Information Systems and Health Care IV: Real-Time ROC Analysis to Evaluate Radiologists' Performanceof Interpreting Mammograpny

This paper describes how to use Receiver Operator Characteristic (ROC) analysis to evaluate radiologists’ performance of interpreting digital mammograms in real-time. We developed an experimental testing system, which implemented a set of clinical lesion-matching rules to prepare raw ROC data. The system can automatically provide detailed evaluations of the performance, such as sensitivity, specificity, positive predictive value, negative predictive value, diagnostic accuracy, ROC curve, and area under the curve (Az). Based on a preliminary evaluation of the system, we found that ROC analysis is appropriate for a real-time computer application, directly using the raw data from a database, to evaluate the performance of radiology residents.